Gray Swan cofounders Zico Kolter and Matt Fredrikson explain why AI systems need a different security mindset, focusing on indirect prompt injection, agent vulnerabilities and correlated failures. They walk through automated red teaming tools like Shade and the Gray Swan Arena, discuss guardrails, and argue that bigger models aren’t inherently safer and require bespoke security, identity management, and compliance measures.

• Human red-teamers ranked only fourth in robustness testing against browser-based agents, behind specialized automated red-teaming models.
• Scaling model size doesn't automatically improve safety, and agents introduce new vulnerabilities distinct from traditional IT security risks.
• The "lethal trifecta" (untrusted data, private data, exfiltration paths) creates attack surfaces that make a major prompt-injection breach feel inevitable.
• Effective AI defense will require machine-driven interpreters and agent-native identity/permissions systems, since humans can't keep pace with automated attack tools.